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of 

A Comparison of Individual Ponding Improvement Scoroa on n Group 36 Community 
Collogo Freshmen na Monaurod by tbo Crudo Gain Mothod and the 

Poaidual Gain Mothod by Horat Gorard Taachow.Ph.D. 

Moat of tho provioua roaoareh invoatigating galna in roading ovor a 
doaignatod poriod of tiao roporta on two methods oaplo yod to aoaauro 
group or individual roading inprovomont. Tho two aothoda appoar to >e vary 
aiailar in thoir procoduro for both apply aiaplo subtraction of pro-tost 
acoroa froa poat-toat scores. Tho firat aothod of aeaauring roading gains 
subtracts tho pro-tost acoroa from tho poat-toat acoroa and calls tho 
difforoncos botwoon tho two accrooi ’’crudo gain”. Symbolically * tho formula 
roprosonting crudo gain iaj Crudo gain « X 2 - Xj. Tho aocond aothod is 
basod on tho saao principlo of finding tho difforonco botwoon poat-toat 
and pre-teat scoros but expresses tho gain aa a porcontago of tho 
initial or pro-toat acoroa and la thoroforo called ’’percent gain”* 
Symbolically, percent gain is represented by tho formula* Percent gain * 

X 2 - Hy/ Xj . 
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Furthermore , research on gains in reading shove that apparently 
negative correlations resulted between Initial and final status in 
reading improvement when measured by either the crude gain method or 
the percent gain method as evidenced by Ranson ( 7) , Bloomer ( 1 ) , Kamman(3 ) 
9 Chansky auid Bregman(2) t Ramsey (5 ) , and Schneyer(8)* 

Reviewing crude gain studies Manning and DuBols(M observed that 
the negative correlations in these reading studies may simply refleot 
the spuriously negative correction between initial status and crude gair. 
"These difficulties"; commented Manning and DuBois, "which are encountered 
in measuring change by the absolute difference between two test scores, 
would probably be overcome if there were assurance that increments in 
scores on both initial and final test were equivalent, that the two teats 
have the same hero point, and the tests are valid and reliable at all 
levels of proficiency" ( *, p.290). These assumptions, however, will 

ordinarily not be tenable* 

Orison of the study 

This study began in an effort to overcome the conventional way of 
measuring reading gains of college freshmen and sophomores by either the 
crude gain or the percent gain method# This effort was also encouraged 
as well as stimulated by the observations that 

(1) students whose scores were lower on the pre-tests made apparently 
gnttir gains as measured by the post-tests than did students with 
average and above average pre-test scores, and (2) students whose scores 
were higher on the pre-tests made apparently less progress as measured 
by the post-tests than did students with lower pre-test scores* 

For example, a student with a lower pre-test score may aehieve a 
crude gain of +10 units as measured by the post-test score minus the 
pre-test score, while, at the same time and on the same test, a student 
with a higher initial score may achieve +3 units of crude gain as 
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measured by the post-test minus pre-test soorss. It mould tbtn appear 
thut according to 5obnayar ( % ) poor raadara aro bonofitod moro by reading 
training than ara good raadars. Such statement seemed not only te contradict 
oommon sense but also to discredit psychological expectation* 

Purpose of the study 

The purpose of this atudy mas to compare reading gains made by 36 
college freshmen as measured by the crude gain method and the residual 

method. The study raised the following questions} 

1. Did the results of the individual reading gains made in vocabulary, 
comprehension, total reading, and reading rate as measured by the crude gain 
method differ vhen the same reading gains mere measured by the residual 
gain method? 

2* How did the position of the aero line or no-leaming line 
affect individual reading gains in vocabulary, comprehension, total 
reading, and reading rate? 

3. How did the peatttan of the regression line affect the same gains 
made in the same sub-tests? 

aalgMBi 

This study used as a criterion measure before and after instruction 
the Nelson-Denny leading Test, Revised, Form A and B* To compute crude 
gains as well as residual gains of the 33 college freshmen the raw scores 
of the Helson-Denny Reading Test were used. Since the raw scores are 
free from concomitant considerations of norming samples, educational grade 
levels, age, and sex, they were used in preference to jiercentile norms and 
grade equivalent norms in order that all students would be on an equal basis. 

To compute individual reading improvement by the crude gain method, 
students 9 pre-teat scores were subtracted from their post- test scores, that 
is, the and the final status have been treated according to the 
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fonmla? Crude gain. X 2 . x r Graphically, oruda gain raaulta «n plotted 
against each other on the X and I ares in relationship to the nero line, 
y » x. Considering residual gain statistics, both the computational and 
the graphical nethod were applied to determine individual gains in vocabulary, 

conprehention, total reading, and reading rate. 

To compute residual gain, the raw score formula has been used(6)j 

T,X * T «|jX + Cl where X * pre-test score, I * post-test score, C • I • bX, 

v . K £ xx * T) j a post-test mean, and X « pre-test mean, 

^2 (cX)2 

n .j . t th. graphical method for ..tlmmtlng raaldnal gain, tb. following 

formula baa baan applied: f» « bX ♦ C, whara T' « predicted poet-tart aeora, 

b z " (SX) (IT) f x « arbitrary pra-taat aoora value, and « • I - bX, 

* JJSX^ — (JX)^ 

The computational method shows the predicted regression and the residual 
gala of the post-test scores on the pre-test scores. The graphical method 
Shows the regression line based on the data of the computational method 
for the post-test scores on the pre-test scores. 

The assessment of individual reading gains on the Nelson-Denny sub-tests 
of the 36 college freshmen is presented by the computational method in 
Table 1 and the graphical method in Table 2, Figures 1 to 4. 

The computational method shows (1) the crude gains based on the simple 
differences between the pre-test and the post-test raw scores And (2) the 
residual gain method based on the predicted regressions of the post-test 

raw scores on the pre-test raw scores, 

Tb, graphical p» Mutation of the crud. gain method 1. bawd on y -x 
uhlob fear tb. purpose of discussion 1. called tb. *.ro Un. or no-learning 
lin. as shown In Table 2, Pig. 1 to 4. Progress In stud.nt'a proficiency 1. 
noted by a plus sign be 4^ crude gain number and with ite position 



Table 1: Computation of erode and residual gain of 
individual student reading improtement 
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40 


R.R. 


150 


226 
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11 


16 
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16 


30 


T.R. 


27 


46 


R.R. 


150 


195 


3 V 


11 


10 


C 


16 


38 


T.R. 


27 


48 


R.R. 


174 


438 


4 V 


15 


17 


C 


12 


36 


T.R. 


27 


53 


R.R. 


262 


599 


5 V 


14 


11 


C 


14 


22 


T.R. 


28 1 


33 


^ R.R. 


174 


299 


,6V 


15 


27 


C 


20 


32 


T.R. 


35 


59 


R.R. 


207 


413 


7 V 


18 


33 


C 


18 


24 


i T.R. 


36 


57 


! R.R. 

1 


185 


379 


i 8 V 


7 


33 


C 


30 


40 


T.R. 


37 


73 


R.R. 


250 


425 


9 V 


15 


26 


. C 


22 


46 


T.R. 


37 


72 


R.R. 


115 


425 


10 V 


21 


37. 
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16 


38 


T.R. 


37 


75 


R.R. 


161 


403 
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-174. 493 
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33.287 


♦ 4.712 


♦ 21 


50.135 


- 2.135 


♦264 


388. 802 
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♦ 2 


22.300 


. 5.300 


♦ 24 


31.634 


♦ 4.365 


♦ 26 


50.135 


♦ 2. 864 


+337 


459.601 


♦ 139.398 


- 3 


21.261 


- 10.261 


♦ 8 


32. 460 


- 10.460 


! ♦ 5 


50.923 


- 17.923 


j +125 


388. 802 


- 89.802 


♦ 12 


22. 300 


♦ 4.699 


+ 12 


34. 939 


- 2.939 


♦ 24 


56.438 


♦ 2. 561 


♦206 


415.352 


. 2.352 


♦ 15 


25.417 


♦ 7. 582 


♦ 6 


34.113 


- 10.113 


♦ 21 


57.226 


<0 

N 

CM 

• 

• 


♦194 


397.652 


- 18.652 


♦ 26 


13. 987 


♦ 19.012 


♦ 10 


39.071 


00 

CM 
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• 

♦ 


♦ 36 


58.014 


♦ 14.985 


i +175 


449. 947 


. 24. 947 


1 ♦ " 


22.300 


+ 3.699 


♦ 24 


35.766 


♦ 10.233 


♦ 35 


58.014 


♦ 13.985 


♦310 


1341.334 


♦ 83.665 


♦ 16 


28. 534 


♦ 8.465 


♦ 22 


33.287 


♦ 4.712 


♦ 38 


58.014 


♦ 16.985 


♦242 


378.343 


♦ 24.656 
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Table 1 continued 
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38 


49 


♦ 11 


58. 802 


- 
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R.R. 


150 
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1 +177 
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- 


42.493 


12 


V 


18 


25 


♦ 7 


25. 417 


- 


.417 




c 


26 


38 


♦ 12 


' 37.418 


♦ 


.581 




T.R. 


44 


63 


♦ 19 


63.530 


- 


.530 




R. R. 


174 


279 


♦105 


388.802 


.109. 802 


13 


V 


19 


26 


♦ 7 


26.456 


- 


.456 




c 


26 


42 


♦ 16 


37.418 
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4.581 




T.R. 


45 


68 


♦ 23 


64.318 
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R.R. 


250 


615 


♦365 
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14 


V 


30 


35 
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16 


34 
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T.R. 


46 
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27 
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49 
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24 


31 
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26 
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T.R. 


50 


73 


♦ 23 
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♦ 
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R.R. 


150 
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V 


27 


34 


+ 7 
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.769 


c 


24 


36 


♦ 12 


36. 592 
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T.R. 


51 


70 


♦ 19 
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R.R. 
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356 


+195 
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18 


V 


20 


34 


♦ 14 
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c 


32 


44 


♦ 12 
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T.R. 


52 


78 


♦ 26 


69.834 


♦ 
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R.R. 


94 


319 


+225 


324.439 


- 


5.439 


19 


V 


22 


39 


+ 17 


29.573 


♦ 


9.426 




c 


30 


36 


♦ 6 


39.071 


- 


3. 071 i 




T.R. 


52 


75 


; + 23 


69. 834 


+ 


5. 165 : 




R.R. 


226 


450 


1 +224 
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♦ 


19.361 



28 

48 

76 

344 



- 1 
4 24 
4 23 
4 94 



36.847 

36.592 

70.622 

449.947 



- 8.847 
+ 11.407 
♦ 5.377 

-105. 947 
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V 


23 
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c 


30 


40 
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T.R. 


53 


67 


♦ 14 


70. 622 
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R.R. 


287 


403 


♦ 116 


479.715 
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22 


V 


31 


35 


+ 4 j 


38. 925 


- 3.925 




C 


26 


32 


♦ 6 1 


37. 418 


. 5. 418 




T.R. 


57 


67 


♦ 10 


73.773 
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R.R. 


384 


511 


♦ 127 


557.755 


- 46.755 


23 


V 


26 


38 


♦ 12 


33.730 


♦ 4.269 




C 


32 


34 


♦ 2 


39.897 
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T.R. 


58 


72 


♦ 14 
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V 
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♦364 
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Table 2s graphical presentation of crude gain and residual gain for vocabulary, comprehension 
to tal reading, and reading rate of Individual student reading improvement* 
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Figure 1. Regression line, vocabulary - Figure 2. Regression line, comprehension 
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on the graph above tho aero line; no learning progroaa la indicatedby *Q and 
with lta position directly on tho zero line; and failure of lmprovoaont la 
mmrkod by a ainua sign be fort tho crude gain numbor and with lta position 
below tho zero lino. 

Tho graphical method of tho residual gain procedure shows tie regression 
lino determined by its intercept and its slope* At any given value of X, the 
hlght of the regression line tells the average value of Y. Based onthe linear 
regression formula Y« a + bn, where x is aero, y is equal to a which indicates 
where the line intercepts the Y«axis, and b, the slope or regression coefficient 
indicates how much Y changes with a unit change in x. A student's residual gain 
is found by looating the intersecting lines on the graph fob the student's 
X and Y scores* The meeting point of the intersecting lines falls either on 
or above or below the regression line* If the point which represents the 
student's performance falls directly on the line, the student has performed 
as predicted* If the point falls above the regression line, the distance 
between its position and the regression line indicates that the sfci dent's 
performance was higher than predicted on the basis of the pre-test score 
and a plus sign precedes the residual gain score* If the point falls below 
the regeresslon line, the distance between its position and the regression 
line indicates that the student's performance was lower than predicted on 
the basis of his pre-test scores and a minus sign precedes his residual gain 
so or*. Thu., « minus sl«n do*, not «P*U o* improv«*ht, *»t Mllrt 

that the student did not yet improve as much as was expected upon prediction* 

For instance, student No. 7 in Table 1 received a pre-course vocabulary 
raw score of 18 and a post— course raw score of 33 which denotes a crude gain 
of +13* When considering the same pre-and post-test raw scores his gain as 
predicted was 47*582 uhits above the regression line as Shown in Table2, Fig«1* 

The same student's performance in comprehension showed a crude gain of 
+g quits but according to prediction full further behind by -10*113 units 



PW « 



(Table 1). In relationship to the regression line he plaeed below 
as shown in Table 2, Fig. 2. In total reading his Improvement in crude 
gtdn was 421 units, but when measured by the residual gain procedure 
the saae total reading perfornance was by *.226 units (Table 1) lower than 
it should have been according to prediction and his perfornance placed 
therefore below the regression line(Table 2,Fig.3). The crude gain in 
his reading rate perfornance was -H04 units, while the residual gain 
upon prediction indicated -18.652 units (Table 1) below the regression 
Hw (Table 2, Fig. 4). In conclusion, the student's No. 7 perfomances in 
comprehension, total reading, end reading rate have not improved to the 
degree as they lere predicted on the basis of final on initial scores. 

Contrasting the student's residual gains with the crude gains (Table 1, 
No. 7), it appears that his exude gain assessment denotes substantial but 
erroneous overestimation of his reading advancement. 

Crude sain versus residual gain 

To analyze the effects of crude gain and residual gain on differences 
in Individual proficiency the reading performances in vocabulary of ten 
college freshmen were seleoted from Table 1 . These results are listed in 




yx= 6.7145 + 1.0390 
x=y 
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At inspection of Table 3 the numerical unite of crude gain with 
those of residual gain are compared* The graph shows the crude gain unite 
in relationship to the aero line as well as the residual gain units in 
relationship to the regression line. To assess crude gain or residual gain 
units on the graph, it is necessary to count either upward or downward 

frow the aero line or the regression line. 

What message do the results of crude gain compared with the results 

of residual gain convey to the reading instructor in regard to the 
students individual reading improvement? In eight cases out of ten crude 
gain tended to overestimate individual reading proficiency and in two cases 
tended to underestimate individual readin^proficiency. In the eitfit cases 
with the exception of hut two -Wo. 5 and 8- crude gains indicated that each 
student has advanced further in his reading proficiency than he actually did 
While in the remaining two cases crude gains Indicated that each student has 
fallen less behind in his reading proficiency than he actually did on 
prediction of residual gains. 

Further investigation of the effects of crude gain and residual gain, 



Shows a comparison of four comprehension scores in Table 4. 



m isUl 



No. in 

i 

8 

29 

30 



Crude Gain Residual 

- £ 2 & — 

♦10 -7.287 

+10 ♦ .828 

+10 +3.275 

♦10 +8.788 



Belying on orude gain reunite each student appears to have uads the ease 
mount of progress expressed In units and therefore may receive the aaae 
grade designation of, for Instance, B. Summing, hoeever, the residual gains 
upon prediction of the same four couppehenslon scores, the Individual gain 
units appear to he different frou the apparent uniform crude gain units. 
Applying grade designations in accordance with the residual gains studant 
Bo.1 may receive a D, student lto.2 a C, while students No.3 and * may 
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receive * B and an A respectively. 

Comparing atudant performances of crude gala with residual gain in 

total reading, alx eonparloona are demonstrated in Table 5. 

Tattl# 5 No. in Crudt Residual 

TaM .0 1 Aain gain 

3 +21 -2.135 

7 +21 . .225 

13 +23 +3.581 

15 +23 +4.741 

19 +23 +5.155 

20 +23 +5.377 

Crudo gain of +21 units for students No .3 and 7 seams to overestimate again 
individual proficiency in total re aging by +18.855 units and by +20.774 units 
respectively. Residual gain procedure instead seems to warn the reading 
instructor that the reading performances of students No .3 and 7 have 
apparently fallen further behind by .2.135 units and by ..225 units in each 
case and that the two students have not achieved as predicted. 

The following four scores in Table 5 show a crude gain of +23 units. 

While each students performance in total reading appears to be equal to the 
other when oeasured by the crude gain method, no such equal appearance of gain 
can be olaiaod when the individual total reading improvement is evaluated 
in the light of the residual gain procedure. According to crude gain measurement 
students who started out with lower pre.test scores tended to Improve as much 
as or more than those students with higher initial scores. Measuring, 
however, total reading improvement by the residual gain procedure a more 
realistic and reliable advancement is the answer. Students who started with 
a hig her twitiai status tended to improve in accordance with their own 
individual proficiency, because residual gain is the deviation of final 
scores from the regression line of fi n a l on initial scores. 



The comparison of the effects of crude gain with those of residual 
gsln in individual readlngjuaprovement as measured on the Nelson-Dtnny 
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Heading Test, Revised, Form A and B, in vocabulary, comprehension, total 
reading, and reading rate nay perait the following conclusions to be drawn: 
Crude gain tends to 

1. overestimate or underestimate individual reading improvement, 

2. Ignore individual differences in the Initial status, 

3 # give cause to faulty application of grades attached to individual 
reading performance, and 

4* introduce erroneous or fallacious impressions of individual advancement 
in reading proficiency. 

In contrast to the crude gain method of measuring individual gain, 
residual gain procedure tends 

1 # to be a more reliable measurement of individual gain in aost reading 
situations, 

2. not to affect the measure of gain in spite of Ini t i a l differences 
on the pre-test scores, 

3. to provide a more realistic basis for attaching grades to individual 
reading improvement, and 

4. to estimate the inferior and the superior improvers in accordance with 
their own proficiency and progress in the improvement of reading. 
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